<html><head>
      <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <title>The Apache HBase Book</title><link rel="stylesheet" href="css/freebsd_docbook.css" type="text/css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.75.2"><meta name="description" content="This is the official book of Apache HBase, a distributed, versioned, column-oriented database built on top of Apache Hadoop and Apache ZooKeeper."><link rel="home" href="book.html" title="The Apache HBase Book"><link rel="next" href="preface.html" title="Preface"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">The Apache HBase
    Book</th></tr><tr><td width="20%" align="left">&nbsp;</td><th width="60%" align="center">&nbsp;</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="preface.html">Next</a></td></tr></table><hr></div><div class="book" title="The Apache HBase Book"><div class="titlepage"><div><div><h1 class="title"><a name="d611e2"></a>The Apache <a class="link" href="http://www.hbase.org" target="_top">HBase</a>
    Book</h1></div><div><p class="copyright">Copyright &copy; 2010 Apache Software Foundation</p></div><div><div class="revhistory"><table border="1" width="100%" summary="Revision history"><tr><th align="left" valign="top" colspan="3"><b>Revision History</b></th></tr><tr><td align="left">Revision 
          0.90.6
        </td><td align="left"></td><td>&nbsp;</td></tr><tr><td align="left" colspan="3">Adding first cuts at Configuration, Getting Started, Data Model</td></tr><tr><td align="left">Revision 
          0.89.20100924
        </td><td align="left">
        5 October 2010
        </td><td align="left">stack</td></tr><tr><td align="left" colspan="3">Initial layout</td></tr></table></div></div><div><div class="abstract" title="Abstract"><p class="title"><b>Abstract</b></p><p>This is the official book of
    <a class="link" href="http://www.hbase.org" target="_top">Apache HBase</a>,
    a distributed, versioned, column-oriented database built on top of
    <a class="link" href="http://hadoop.apache.org/" target="_top">Apache Hadoop</a> and
    <a class="link" href="http://zookeeper.apache.org/" target="_top">Apache ZooKeeper</a>.
      </p></div></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="preface"><a href="preface.html">Preface</a></span></dt><dt><span class="chapter"><a href="getting_started.html">1. Getting Started</a></span></dt><dd><dl><dt><span class="section"><a href="getting_started.html#d611e81">1.1. Introduction</a></span></dt><dt><span class="section"><a href="quickstart.html">1.2. Quick Start</a></span></dt><dd><dl><dt><span class="section"><a href="quickstart.html#d611e97">1.2.1. Download and unpack the latest stable release.</a></span></dt><dt><span class="section"><a href="quickstart.html#start_hbase">1.2.2. Start HBase</a></span></dt><dt><span class="section"><a href="quickstart.html#shell_exercises">1.2.3. Shell Exercises</a></span></dt><dt><span class="section"><a href="quickstart.html#stopping">1.2.4. Stopping HBase</a></span></dt><dt><span class="section"><a href="quickstart.html#d611e254">1.2.5. Where to go next</a></span></dt></dl></dd><dt><span class="section"><a href="notsoquick.html">1.3. Not-so-quick Start Guide</a></span></dt><dd><dl><dt><span class="section"><a href="notsoquick.html#requirements">1.3.1. Requirements</a></span></dt><dt><span class="section"><a href="notsoquick.html#standalone_dist">1.3.2. HBase run modes: Standalone and Distributed</a></span></dt><dt><span class="section"><a href="notsoquick.html#example_config">1.3.3. Example Configurations</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="upgrading.html">2. Upgrading</a></span></dt><dd><dl><dt><span class="section"><a href="upgrading.html#upgrade0.90">2.1. Upgrading to HBase 0.90.x from 0.20.x or 0.89.x</a></span></dt></dl></dd><dt><span class="chapter"><a href="configuration.html">3. Configuration</a></span></dt><dd><dl><dt><span class="section"><a href="configuration.html#hbase.site">3.1. <code class="filename">hbase-site.xml</code> and <code class="filename">hbase-default.xml</code></a></span></dt><dd><dl><dt><span class="section"><a href="configuration.html#hbase_default_configurations">3.1.1. HBase Default Configuration</a></span></dt></dl></dd><dt><span class="section"><a href="hbase.env.sh.html">3.2. <code class="filename">hbase-env.sh</code></a></span></dt><dt><span class="section"><a href="log4j.html">3.3. <code class="filename">log4j.properties</code></a></span></dt><dt><span class="section"><a href="important_configurations.html">3.4. The Important Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="important_configurations.html#required_configuration">3.4.1. Required Configurations</a></span></dt><dt><span class="section"><a href="important_configurations.html#recommended_configurations">3.4.2. Recommended Configuations</a></span></dt></dl></dd><dt><span class="section"><a href="client_dependencies.html">3.5. Client configuration and dependencies connecting to an HBase cluster</a></span></dt><dd><dl><dt><span class="section"><a href="client_dependencies.html#d611e2092">3.5.1. Java client configuration</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="shell.html">4. The HBase Shell</a></span></dt><dd><dl><dt><span class="section"><a href="shell.html#scripting">4.1. Scripting</a></span></dt><dt><span class="section"><a href="shell_tricks.html">4.2. Shell Tricks</a></span></dt><dd><dl><dt><span class="section"><a href="shell_tricks.html#d611e2178">4.2.1. <code class="filename">irbrc</code></a></span></dt><dt><span class="section"><a href="shell_tricks.html#d611e2196">4.2.2. LOG data to timestamp</a></span></dt><dt><span class="section"><a href="shell_tricks.html#d611e2214">4.2.3. Debug</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="build.html">5. Building HBase</a></span></dt><dd><dl><dt><span class="section"><a href="build.html#mvn_repo">5.1. Adding an HBase release to Apache's Maven Repository</a></span></dt></dl></dd><dt><span class="chapter"><a href="build.html">6. Developers</a></span></dt><dd><dl><dt><span class="section"><a href="build.html#ides">6.1. IDEs</a></span></dt><dd><dl><dt><span class="section"><a href="build.html#eclipse">6.1.1. Eclipse</a></span></dt></dl></dd><dt><span class="section"><a href="unit.tests.html">6.2. Unit Tests</a></span></dt><dd><dl><dt><span class="section"><a href="unit.tests.html#mockito">6.2.1. Mocito</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="mapreduce.html">7. HBase and MapReduce</a></span></dt><dd><dl><dt><span class="section"><a href="mapreduce.html#splitter">7.1. The default HBase MapReduce Splitter</a></span></dt><dt><span class="section"><a href="mapreduce.example.html">7.2. HBase Input MapReduce Example</a></span></dt><dt><span class="section"><a href="mapreduce.htable.access.html">7.3. Accessing Other HBase Tables in a MapReduce Job</a></span></dt><dt><span class="section"><a href="mapreduce.specex.html">7.4. Speculative Execution</a></span></dt></dl></dd><dt><span class="chapter"><a href="schema.html">8. HBase and Schema Design</a></span></dt><dd><dl><dt><span class="section"><a href="schema.html#schema.creation">8.1. 
      Schema Creation
  </a></span></dt><dt><span class="section"><a href="number.of.cfs.html">8.2. 
      On the number of column families
  </a></span></dt><dt><span class="section"><a href="timeseries.html">8.3. 
  Monotonically Increasing Row Keys/Timeseries Data
  </a></span></dt><dt><span class="section"><a href="keysize.html">8.4. Try to minimize row and column sizes</a></span></dt><dt><span class="section"><a href="schema.versions.html">8.5. 
  Number of Versions
  </a></span></dt></dl></dd><dt><span class="chapter"><a href="hbase_metrics.html">9. Metrics</a></span></dt><dd><dl><dt><span class="section"><a href="hbase_metrics.html#metric_setup">9.1. Metric Setup</a></span></dt><dt><span class="section"><a href="rs_metrics.html">9.2. RegionServer Metrics</a></span></dt><dd><dl><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.blockCacheCount">9.2.1. <code class="varname">hbase.regionserver.blockCacheCount</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.blockCacheFree">9.2.2. <code class="varname">hbase.regionserver.blockCacheFree</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.blockCacheHitRatio">9.2.3. <code class="varname">hbase.regionserver.blockCacheHitRatio</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.blockCacheSize">9.2.4. <code class="varname">hbase.regionserver.blockCacheSize</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.compactionQueueSize">9.2.5. <code class="varname">hbase.regionserver.compactionQueueSize</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsReadLatency_avg_time">9.2.6. <code class="varname">hbase.regionserver.fsReadLatency_avg_time</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsReadLatency_num_ops">9.2.7. <code class="varname">hbase.regionserver.fsReadLatency_num_ops</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsSyncLatency_avg_time">9.2.8. <code class="varname">hbase.regionserver.fsSyncLatency_avg_time</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsSyncLatency_num_ops">9.2.9. <code class="varname">hbase.regionserver.fsSyncLatency_num_ops</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsWriteLatency_avg_time">9.2.10. <code class="varname">hbase.regionserver.fsWriteLatency_avg_time</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.fsWriteLatency_num_ops">9.2.11. <code class="varname">hbase.regionserver.fsWriteLatency_num_ops</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.memstoreSizeMB">9.2.12. <code class="varname">hbase.regionserver.memstoreSizeMB</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.regions">9.2.13. <code class="varname">hbase.regionserver.regions</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.requests">9.2.14. <code class="varname">hbase.regionserver.requests</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.storeFileIndexSizeMB">9.2.15. <code class="varname">hbase.regionserver.storeFileIndexSizeMB</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.stores">9.2.16. <code class="varname">hbase.regionserver.stores</code></a></span></dt><dt><span class="section"><a href="rs_metrics.html#hbase.regionserver.storeFiles">9.2.17. <code class="varname">hbase.regionserver.storeFiles</code></a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="cluster_replication.html">10. Cluster Replication</a></span></dt><dt><span class="chapter"><a href="datamodel.html">11. Data Model</a></span></dt><dd><dl><dt><span class="section"><a href="datamodel.html#conceptual.view">11.1. Conceptual View</a></span></dt><dt><span class="section"><a href="physical.view.html">11.2. Physical View</a></span></dt><dt><span class="section"><a href="table.html">11.3. Table</a></span></dt><dt><span class="section"><a href="row.html">11.4. Row</a></span></dt><dt><span class="section"><a href="columnfamily.html">11.5. Column Family</a></span></dt><dt><span class="section"><a href="cells.html">11.6. Cells</a></span></dt><dt><span class="section"><a href="versions.html">11.7. Versions</a></span></dt><dd><dl><dt><span class="section"><a href="versions.html#versions.ops">11.7.1. Versions and HBase Operations</a></span></dt><dt><span class="section"><a href="versions.html#d611e3050">11.7.2. Current Limitations</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="architecture.html">12. Architecture</a></span></dt><dd><dl><dt><span class="section"><a href="architecture.html#client">12.1. Client</a></span></dt><dd><dl><dt><span class="section"><a href="architecture.html#client.connections">12.1.1. Connections</a></span></dt><dt><span class="section"><a href="architecture.html#client.writebuffer">12.1.2. WriteBuffer and Batch Methods</a></span></dt><dt><span class="section"><a href="architecture.html#client.filter">12.1.3. Filters</a></span></dt></dl></dd><dt><span class="section"><a href="daemons.html">12.2. Daemons</a></span></dt><dd><dl><dt><span class="section"><a href="daemons.html#master">12.2.1. Master</a></span></dt><dt><span class="section"><a href="daemons.html#regionserver.arch">12.2.2. RegionServer</a></span></dt></dl></dd><dt><span class="section"><a href="regions.arch.html">12.3. Regions</a></span></dt><dd><dl><dt><span class="section"><a href="regions.arch.html#arch.regions.size">12.3.1. Region Size</a></span></dt><dt><span class="section"><a href="regions.arch.html#d611e3211">12.3.2. Region Splits</a></span></dt><dt><span class="section"><a href="regions.arch.html#d611e3218">12.3.3. Region Load Balancer</a></span></dt><dt><span class="section"><a href="regions.arch.html#store">12.3.4. Store</a></span></dt></dl></dd><dt><span class="section"><a href="wal.html">12.4. Write Ahead Log (WAL)</a></span></dt><dd><dl><dt><span class="section"><a href="wal.html#purpose.wal">12.4.1. Purpose</a></span></dt><dt><span class="section"><a href="wal.html#wal_flush">12.4.2. WAL Flushing</a></span></dt><dt><span class="section"><a href="wal.html#wal_splitting">12.4.3. WAL Splitting</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="performance.html">13. Performance Tuning</a></span></dt><dd><dl><dt><span class="section"><a href="performance.html#jvm">13.1. Java</a></span></dt><dd><dl><dt><span class="section"><a href="performance.html#gc">13.1.1. The Garbage Collector and HBase</a></span></dt></dl></dd><dt><span class="section"><a href="perf.configurations.html">13.2. Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="perf.configurations.html#perf.number.of.regions">13.2.1. Number of Regions</a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.compactions.and.splits">13.2.2. Managing Compactions</a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.compression">13.2.3. Compression</a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.handlers">13.2.4. <code class="varname">hbase.regionserver.handler.count</code></a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.hfile.block.cache.size">13.2.5. <code class="varname">hfile.block.cache.size</code></a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.rs.memstore.upperlimit">13.2.6. <code class="varname">hbase.regionserver.global.memstore.upperLimit</code></a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.rs.memstore.lowerlimit">13.2.7. <code class="varname">hbase.regionserver.global.memstore.lowerLimit</code></a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.hstore.blockingstorefiles">13.2.8. <code class="varname">hbase.hstore.blockingStoreFiles</code></a></span></dt><dt><span class="section"><a href="perf.configurations.html#perf.hregion.memstore.block.multiplier">13.2.9. <code class="varname">hbase.hregion.memstore.block.multiplier</code></a></span></dt></dl></dd><dt><span class="section"><a href="perf.number.of.cfs.html">13.3. Number of Column Families</a></span></dt><dt><span class="section"><a href="perf.one.region.html">13.4. Data Clumping</a></span></dt><dt><span class="section"><a href="perf.batch.loading.html">13.5. Batch Loading</a></span></dt><dd><dl><dt><span class="section"><a href="perf.batch.loading.html#precreate.regions">13.5.1. 
  Table Creation: Pre-Creating Regions
  </a></span></dt></dl></dd><dt><span class="section"><a href="ch13s06.html">13.6. HBase Client</a></span></dt><dd><dl><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.autoflush">13.6.1. AutoFlush</a></span></dt><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.caching">13.6.2. Scan Caching</a></span></dt><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.selection">13.6.3. Scan Attribute Selection</a></span></dt><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.scannerclose">13.6.4. Close ResultScanners</a></span></dt><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.blockcache">13.6.5. Block Cache</a></span></dt><dt><span class="section"><a href="ch13s06.html#perf.hbase.client.rowkeyonly">13.6.6. Optimal Loading of Row Keys</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="blooms.html">14. Bloom Filters</a></span></dt><dd><dl><dt><span class="section"><a href="blooms.html#bloom.config">14.1. Configurations</a></span></dt><dd><dl><dt><span class="section"><a href="blooms.html#d611e3668">14.1.1. <code class="code">HColumnDescriptor</code> option</a></span></dt><dt><span class="section"><a href="blooms.html#d611e3687">14.1.2. <code class="varname">io.hfile.bloom.enabled</code> global kill
        switch</a></span></dt><dt><span class="section"><a href="blooms.html#d611e3702">14.1.3. <code class="varname">io.hfile.bloom.error.rate</code></a></span></dt><dt><span class="section"><a href="blooms.html#d611e3710">14.1.4. <code class="varname">io.hfile.bloom.max.fold</code></a></span></dt></dl></dd><dt><span class="section"><a href="bloom_footprint.html">14.2. Bloom StoreFile footprint</a></span></dt><dd><dl><dt><span class="section"><a href="bloom_footprint.html#d611e3738">14.2.1. BloomFilter in the <code class="classname">StoreFile</code>
        <code class="classname">FileInfo</code> data structure</a></span></dt><dt><span class="section"><a href="bloom_footprint.html#d611e3766">14.2.2. BloomFilter entries in <code class="classname">StoreFile</code>
        metadata</a></span></dt></dl></dd></dl></dd><dt><span class="chapter"><a href="trouble.html">15. Troubleshooting and Debugging HBase</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.html#trouble.general">15.1. General Guidelines</a></span></dt><dt><span class="section"><a href="trouble.log.html">15.2. Logs</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.log.html#trouble.log.locations">15.2.1. Log Locations</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.tools.html">15.3. Tools</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.tools.html#trouble.tools.searchhadoop">15.3.1. search-hadoop.com</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.tail">15.3.2. tail</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.top">15.3.3. top</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.jps">15.3.4. jps</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.jstack">15.3.5. jstack</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.opentsdb">15.3.6. OpenTSDB</a></span></dt><dt><span class="section"><a href="trouble.tools.html#trouble.tools.clustersshtop">15.3.7. clusterssh+top</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.client.html">15.4. Client</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.client.html#trouble.client.scantimeout">15.4.1. ScannerTimeoutException</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.rs.html">15.5. RegionServer</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.rs.html#trouble.rs.startup">15.5.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.runtime">15.5.2. Runtime Errors</a></span></dt><dt><span class="section"><a href="trouble.rs.html#trouble.rs.shutdown">15.5.3. Shutdown Errors</a></span></dt></dl></dd><dt><span class="section"><a href="trouble.master.html">15.6. Master</a></span></dt><dd><dl><dt><span class="section"><a href="trouble.master.html#trouble.master.startup">15.6.1. Startup Errors</a></span></dt><dt><span class="section"><a href="trouble.master.html#trouble.master.startup">15.6.2. Shutdown Errors</a></span></dt></dl></dd></dl></dd><dt><span class="appendix"><a href="tools.html">A. Tools</a></span></dt><dd><dl><dt><span class="section"><a href="tools.html#hbck">A.1. HBase <span class="application">hbck</span></a></span></dt><dt><span class="section"><a href="apas02.html">A.2. HFile Tool</a></span></dt><dt><span class="section"><a href="wal_tools.html">A.3. WAL Tools</a></span></dt><dd><dl><dt><span class="section"><a href="wal_tools.html#hlog_tool">A.3.1. <code class="classname">HLog</code> tool</a></span></dt></dl></dd><dt><span class="section"><a href="compression.tool.html">A.4. Compression Tool</a></span></dt><dt><span class="section"><a href="decommission.html">A.5. Node Decommission</a></span></dt><dd><dl><dt><span class="section"><a href="decommission.html#rolling">A.5.1. Rolling Restart</a></span></dt></dl></dd></dl></dd><dt><span class="appendix"><a href="compression.html">B. Compression In HBase</a></span></dt><dd><dl><dt><span class="section"><a href="compression.html#compression.test">B.1. CompressionTest Tool</a></span></dt><dt><span class="section"><a href="hbase.regionserver.codecs.html">B.2. 
    <code class="varname">
    hbase.regionserver.codecs
    </code>
    </a></span></dt><dt><span class="section"><a href="lzo.compression.html">B.3. 
    LZO
    </a></span></dt><dt><span class="section"><a href="gzip.compression.html">B.4. 
    GZIP
    </a></span></dt></dl></dd><dt><span class="appendix"><a href="faq.html">C. FAQ</a></span></dt><dt><span class="appendix"><a href="apd.html">D. YCSB: The Yahoo! Cloud Serving Benchmark and HBase</a></span></dt><dt><span class="index"><a href="book_index.html">Index</a></span></dt></dl></div><div class="list-of-tables"><p><b>List of Tables</b></p><dl><dt>11.1. <a href="datamodel.html#d611e2614">Table <code class="varname">webtable</code></a></dt><dt>11.2. <a href="physical.view.html#d611e2698">ColumnFamily <code class="varname">anchor</code></a></dt><dt>11.3. <a href="physical.view.html#d611e2737">ColumnFamily <code class="varname">contents</code></a></dt></dl></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left">&nbsp;</td><td width="20%" align="center">&nbsp;</td><td width="40%" align="right">&nbsp;<a accesskey="n" href="preface.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">&nbsp;</td><td width="20%" align="center">&nbsp;</td><td width="40%" align="right" valign="top">&nbsp;Preface</td></tr></table></div></body></html>